R.H. Thomas and W. Crowther. "The Uniform System: An Approach to Runtime Support for Large

43 processor speeds continue to improve more quickly than memory or interconnection speeds, communication will be an increasing percentage of an application's execution time; scheduling methods that reduce both communication and synchronization overhead are going to have an even greater impact in the future. Anity scheduling simultaneously balances overhead due to synchronization, communication, and load imbalance. Anity scheduling has the load balancing properties of the best dynamic scheduling algorithms, reduces synchronization costs by employing per-processor work queues, and exploits processor anity when it exists. Anity scheduling is robust. Our experiments cover a range of applications with widely varying characteristics. For applications that create anity between iterations and processors, anity scheduling is by far the best algorithm. For applications with input-dependent load imbalance (e.g., transitive closure), anity scheduling is also the best scheduling algorithm. Even for applications that have no anity to exploit, but exhibit signicant potential for load imbalance (e.g., adjoint convolution and L4), anity scheduling is among the best algorithms. In summary, our theoretical and experimental evaluation shows that anity scheduling has the attractive load balancing properties of the best known loop scheduling algorithms, but also reduces communication overhead substantially. This overhead is quite high on current multiprocessors, and is likely to increase in the future. We conclude that loop scheduling techniques, such as anity scheduling, that minimize communication overhead will be increasingly important in the future. Acknowledgements We would like to thank Argonne National Laboratory for allowing us to use their Sequent Symmetry, and Donna Bergmark and the Cornell Theory Center for assistance with and use of their KSR-1. 42 Finally, there is an important aspect to loop scheduling we have not considered: scheduling loops that have dependencies within the statements of a single iteration, across iterations, or both [18, 23]. The diculties in scheduling this type of loop lead to challenging graph-theoretic problems, whose general form is intractable and requires heuristic solutions. Previous approaches to this problem are based on static scheduling, and therefore inherently exploit anity, unlike dynamic loop scheduling algorithms. 7 Conclusions In this paper we argued that the non-uniform access time to data in a shared-memory multiprocessor (due to local caches or memory) introduces a new dimension to the loop scheduling problem: communication overhead. We showed that traditional loop scheduling algorithms, which emphasize load imbalance and synchronization overhead while ignoring communication overhead, impose a signicant performance penalty on parallel applications. We described a new …